Sentiment and behaviour annotation in a corpus of dialogue summaries

This paper proposes a scheme for sentiment annotation. We show how the task can be made tractable by focusing on one of the many aspects of sentiment: sentiment as it is recorded in behaviour reports of people and their interactions. Together with a number of measures for supporting the reliable application of the scheme, this allows us to obtain sufficient to good agreement scores (in terms of Krippendorf's alpha) on three key dimensions: polarity, evaluated party and type of clause. Evaluation of the scheme is carried out through the annotation of an existing corpus of dialogue summaries (in English and Portuguese) by nine annotators. Our contribution to the field is twofold: (i) a reliable multi-dimensional annotation scheme for sentiment in behaviour reports; and (ii) an annotated corpus that was used for testing the reliability of the scheme and which is made available to the research community

Programming fundamentals and human factors: an empirical study of three variables

In the present study we identify and experimentally investigate variations in the values of three important variables that are present in learning environments for programming fundamentals: the type of the source of problems (concrete vs. abstract); the type of the programming language grammar (context-free vs. natural language like); and the distance between the concepts in the source of problems and the programming language primitives (close vs. distant). We understand that the results of our research can be used to design better courses and learning material, to improve students' performance in the learning of introductory programming

No Place For Hate Speech @ AMI: Convolutional Neural Network and Word Embedding for the Identification of Misogyny in Italian

In this article, we describe two classification models (a Convolutional Neural Network and a Logistic Regression classifier), arranged according to three different strategies, submitted to subtask A of Automatic Misogyny Identification at EVALITA 2020. Results were very encouraging for detecting misogyny, even though aggressiveness was less accurate. Our second strategy, consisting of a Convolutional Neural Network and logistic regression to identify misogyny and aggressiveness, respectively, won the sixth place in the competition.In questo articolo, descriviamo due modelli di classificazione (i.e., Convolutional Neural Network e Regressione Logistica), organizzati secondo tre diverse strategie, per il subtask A dello shared task Automatic Misogyny Identification a EVALITA 2020. I risultati sono stati molto incoraggianti nel rilevamento della misoginia, anche se l’aggressività viene riconosciuta con una precisione più basse. La nostra seconda strategia (Convolutional Neural Network per misoginia e Regressione Logistica per aggressività) ci ha permesso di ottenere il sesto posto nella competizione

Introducing a corpus of human-authored dialogue summaries in Portuguese

In this paper, we introduce a corpus of human-authored dialogue summaries collected through a web-experiment. The corpus features (i) one of the few existing corpora of written dialogue summaries; (ii) the only corpus available for dialogue summaries in Portuguese; and (iii) the only available corpus of summaries produced for dialogues whose participants’ politeness alignment was systematically varied. Comprising 1,808 human-authored summaries, produced by 452 summarisers, for four different dialogues, this is, to the best of our knowledge, the largest individual corpus available for dialogue summaries, with the highest number of participants involved

Complementando o Aprendizado em Programação: Revisitando Experiências no Curso de Sistemas de Informação da USP

O curso de Bacharelado em Sistemas de Informação da Universidade de São Paulo trabalha pela constante melhoria na formação que oferece para seus alunos, o que requer um trabalho contínuo de inovação e aprimoramento do processo de ensino-aprendizagem executados por seus professores e alunos. Na busca desta melhoria, os professores e alunos vêm realizando algumas ações, dentre as quais estão as experiências apresentadas neste artigo: as disciplinas de Desafios de Programação e o Campeonato de Programação para Calouros. Ambas estão focadas na complementação do aprendizado de lógica de programação, algoritmos e estruturas de dados -- assuntos difíceis do ponto de vista didático, mas imprescindíveis na formação técnica de qualidade. O presente artigo revisita e estende análises sobre essas experiências

EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

Welcome to EVALITA 2020! EVALITA is the evaluation campaign of Natural Language Processing and Speech Tools for Italian. EVALITA is an initiative of the Italian Association for Computational Linguistics (AILC, http://www.ai-lc.it) and it is endorsed by the Italian Association for Artificial Intelligence (AIxIA, http://www.aixia.it) and the Italian Association for Speech Sciences (AISV, http://www.aisv.it)

Emotion and automatic dialogue summarisation

Orientadores: Ariadne Maria Brito Rizzoni Carvalho, Paul PiwekTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Esta tese apresenta várias contribuições ao campo da sumarização automática de diálogos. Ela fornece evidências em favor da hipótese de que toda vez que um diálogo apresentar um comportamento muito impolido, por um ou mais de seus interlocutores, este comportamento tenderá a ser descrito em seu resumo. Além disso, os resultados experimentais mostraram também que o relato deste comportamento é feito de modo a apresentar um forte viés, determinado pelo ponto de vista do sumarizador. Este resultado não foi afetado por restrições no tamanho do resumo. Além disso, os experimentos forneceram informações bastante úteis com relação a quando e como julgamentos de emoção e comportamento devem ser adicionados ao resumo. Para executar os experimentos, um esquema de anotação multi-dimensional e categórico foi desenvolvido, podendo ser de grande ajuda a outros pesquisadores que precisem classificar dados de maneira semelhante. Os resultados dos estudos empíricos foram usados para construir um sistema automático de sumarização de diálogos, de modo a testar sua aplicabilidade computacional. A saída do sistema consiste de resumos nos quais a informação técnica e emocional, como julgamentos do comportamento dos participantes do diálogos, são combinadas de modo a refletir o viés do sumarizador, sendo o ponto de vista definido pelo usuárioAbstract: This thesis presents a number of contributions to the field of automatic dialogue summarisation. It provides evidence for the hypothesis that whenever a dialogue features very impolite behaviour by one or more of its interlocutors, this behaviour will tend to be described in the dialogue¿s summary. Moreover, further experimental results showed that this behaviour is reported with a strong bias determined by the point of view of the summariser. This result was not affected by constraints on the summary length. The experiments provided useful information on when and how assessments of emotion and behaviour should be added to a dialogue summary. To conduct the experiments, a categorical multi-dimensional annotation scheme was developed which may also be helpful to other researchers who need to annotate data in a similar way. The results from the empirical studies were used to build an automatic dialogue summarisation system, in order to test their computational applicability. The system¿s output consists of summaries in which technical and emotional information, such as assessments of the dialogue participants¿ behaviour, are combined in a way that reflects the bias of the summariser, being the point of view defined by the userDoutoradoDoutor em Ciência da Computaçã